Blog: Managing a Critical Situation with a Team by Umberto Ottavianelli

Managing a Critical Situation with a Team
written by Umberto Ottavianelli

In full health emergency and just before December 31, 2020, I had to manage the ISP/SP exchange migration of a renowned hospital in Italy. From these few lines, we can already guess the complexity of the situation due to the very nature of the customer indicated. The real challenge was to make sure that the disruption was reduced to a minimum. For this reason, I decided to take action before the current SP stopped providing the service. In other words, I made sure that the current SP and the one to which it migrated (this is the ISP/SP where I work) coexisted.

The first activity was carried out on the CED side (central office) where two redundant routers were configured, one master and the other slave, and was interfaced with the customer firewall sharing the same private IP addressing plan of their current ISP/SP. Subsequently, the branch offices were managed on the MPLS network with 100Mb fiber optic technology. In particular, I only managed the configuration of one of these locations and I wrote a template uploaded to the cloud of my ISP/SP to ensure that delivery colleagues implement the same configuration on the remaining peripheral locations.

The migration of these offices was expected in a second step just before Christmas day. During the migration, the customer informed us that he was in a disruption with the VoIP phones. Unfortunately, until that moment, there had never been talk of this service as the customer believed it was in his management and not managed by the current ISP/SP. In this regard, in order not to have to do rollback, the LAN was created to manage VoIP (also on the CED side) and management QoS was implemented both in the router and in the circuit. The customer seemed satisfied so we were ready for the migration of the next locations.

After a few days, just before December 31, 2020, the customer experiences problems when the peripheral office that is with my ISP/SP has to communicate with the ISP/SP currently used. After careful analysis, it was assumed that the VoIP traffic, on the head office side, passed through a firewall that the customer believed he had managed properly. At that point, a new firewall was installed and configured by the SOC colleagues so as to allow only the VoIP LAN. Once the ethernet cables were properly connected, VoIP also started to work properly with great customer satisfaction.

At that point, the hospital IT chief provided us with the current VoIP IP addressing plan used across all sites. Thanks to this data, I modified the template that I uploaded to the cloud (including updated data; these data are indexed by location and when the template matches the reference site, it correctly extrapolates the data). At that point, the update of the network device configurations of the peripheral offices was managed in delivery and the entire network infrastructure was successfully migrated.

In the end, the client wanted to thank me by calling me personally on the office phone! He understood that the challenge was anything but trivial but that the commitment was maximum. After a few days, he sent an e-mail to my manager to congratulate me on my work and on the other people who, together with me, have committed themselves to building the entire network infrastructure. It was a great satisfaction to have completed this job without giving any disservice and with the compliments of the customer.

About the author:
Hi everyone, my name is Umberto Ottavianelli and I am a Senior Network Engineer. I collaborate with the most important Italian ISP and I deal with the design and implementation of CED offices on multivendor technology. I have always been passionate about technology and fascinated by the world of cybersecurity that I study with and for passion and, often, I find it useful when I interact with Firewall specialists or with the SOC guys.


Interested in getting your story out to our Informed Defenders community?
Visit our Guest Blogger Challenge here.

Way to stay calm under pressure.

2 Likes

Hello @chris.russell, you are definitely right!

Thank you so much AttackIQ for publishing my blog! :blush:

1 Like

Awesome, keep it up.

2 Likes

Congratulations @umberto.ottavianelli.1 what a great project to work on, they don’t come around that often.

2 Likes

Nice one…:+1: Very good

1 Like

Thanks! This is informative. :+1:

1 Like

thanks! good read :slight_smile:

1 Like

Hi guys and thanks to all of you who have read my blog! I am really pleased to know that my blog has been appreciated! I apologize for the delay in replying but I did not notice your warm and appreciated comments!

@chris.russell True! This is the way and it is not always easy to stay calm; the more you keep it, the more lucid you are! I agree with you! :+1: Thanks!

@fanwell.sibanda Sure man! I’ll try! Many thanks! :slightly_smiling_face:

@andrew.nash First off: congratulations on becoming the new Defender of the Month! :clap::clap::clap::handshake:Thank you for your appreciation! Yes, it was actually a good project and the satisfactions were not lacking! Thanks again! :hugs:

@rsh47 I’m glad you enjoyed! Thanks so much! :slightly_smiling_face:

@jerome.luz Thanks to you! I really appreciate that you find it informative! :slightly_smiling_face:

@lindbergh.caldeira Thank you very much for reading it! :blush:

1 Like

Awesome article, thanks for sharing

1 Like

Hello @randy.varela.cordero, I’m glad you appreciated it! Thanks to you for reading it and for the welcome comment! :slightly_smiling_face:

1 Like

I have always admired leaders who don’t act like they are the smartest in the room, but yet understand the very basic of managing the human complexity.

1 Like

.Appreciation is the highest form of prayer, for it acknowledges the presence of good wherever you shine the light of your thankful thoughts

1 Like

Hello @mjohnson, thank you for your excellent observation! I, after all, think just like you!

Hello @nikhil.y, very true! Appreciation, then, is good for our life and makes us feel part of something bigger! A gear that, together with the others, makes everything move. Thank you!

Awesome blog entry @sokkaUMB!

1 Like

Thank you @mariya.gul.khand, you are very kind! :blush:

very well written blog

1 Like

good insights here in the topic

1 Like

Thanks @umberto.ottavianelli.2 This was great as expected.

1 Like

one of the finest blogs Ive ever read!

1 Like